default reasoning
Generics and Default Reasoning in Large Language Models
Kirkpatrick, James Ravi, Sterken, Rachel Katharine
This paper evaluates the capabilities of 28 large language models (LLMs) to reason with 20 defeasible reasoning patterns involving generic generalizations (e.g., 'Birds fly', 'Ravens are black') central to non-monotonic logic. Generics are of special interest to linguists, philosophers, logicians, and cognitive scientists because of their complex exception-permitting behaviour and their centrality to default reasoning, cognition, and concept acquisition. We find that while several frontier models handle many default reasoning problems well, performance varies widely across models and prompting styles. Few-shot prompting modestly improves performance for some models, but chain-of-thought (CoT) prompting often leads to serious performance degradation (mean accuracy drop -11.14%, SD 15.74% in models performing above 75% accuracy in zero-shot condition, temperature 0). Most models either struggle to distinguish between defeasible and deductive inference or misinterpret generics as universal statements. These findings underscore both the promise and limits of current LLMs for default reasoning.
The 2005 AAAI Classic Paper Awards
Haussler's paper was therefore important in linking the new PAC learning theory work with the ongoing work on machine learning within AI. Twenty years later that link is firmly established, and the two research communities have largely merged into one. In fact, much of the dramatic progress in machine learning over the past two decades has come from a fruitful marriage between research on learning theory and design of practical learning algorithms for particular problem classes. Mitchell and Levesque provide commentary on the two AAAI Classic Paper awards, given at the AAAI-05 conference in Pittsburgh, Pennsylvania. The two winning papers were "Quantifying the Inductive Bias in Concept Learning," by David Haussler, and "Default Reasoning, Nonmonotonic Logics, and the Frame Problem," by Steve Hanks and Drew Mc-Dermott.
AAAI Workshop on Non-Monotonic Reasoning
Default and auto-epistemic reasoning were also well represented, with a number of papers discussing aspects, applications, and implementations of default reasoning systerns. Several papers emphasized nonmonotonic facets of computational vision, natural language understanding, and conimo1i-sense reasoning. Thursday evening, a panel discussion was held, with John McCarthy, Dana Scott, and Richmond Thomason as panelists. Compare it with a merely COMMON LISP (Golden Common Lisp@ Version 1.OO): Golden Common Lisp is a registered trademark of Gold Hill Computers. Our low-key, dignified approach to matchingquality candidates with quality companies will offer you the opportunity to examine your alternatives in a confidential, systematic fashion Openingsarenationwide.
Formalizing Deceptive Reasoning in Breaking Bad: Default Reasoning in a Doxastic Logic
Licato, John (Indiana University and Purdue University, Fort Wayne)
The rich expressivity provided by the cognitive event calculus (CEC) knowledge representation framework allows for reasoning over deeply nested beliefs, desires, intentions, and so on. I put CEC to the test by attempting to model the complex reasoning and deceptive planning used in an episode of the popular television show Breaking Bad. CEC is used to represent the knowledge used by reasoners coming up with plans like the ones devised by the fictional characters I describe. However, it becomes clear that a form of nonmonotonic reasoning is necessary—specifically so that an agent can reason about the nonmonotonic beliefs of another agent. I show how CEC can be augmented to have this ability, and then provide examples detailing how my proposed augmentation enables much of the reasoning used by agents such as the Breaking Bad characters. I close by discussing what sort of reasoning tool would be necessary to implement such nonmonotonic reasoning.
1 Partial Models and Non-monotonic Inference K. Konolige
In this paper we will be concerned with such reasoning in its most general form, that is, in inferences that are defeasible: given more information, we may retract them. The purpose of this paper is to introduce a form of non-monotonic inference based on the notion of a partial model of the world. We take partial models to reflect our partial knowledge of the true state of affairs. We then define non-monotonic inference as the process of filling in unknown parts of the model with conjectures: statements that could turn out to be false, given more complete knowledge. To take a standard example from default reasoning: since most birds can fly, if Tweety is a bird it is reasonable to assume that she can fly, at least in the absence of any information to the contrary. We thus have some justification for filling in our partial picture of the world with this conjecture. If our knowledge includes the fact that Tweety is an ostrich, then no such justification exists, and the conjecture must be retracted.
A Plausibility Semantics for Abstract Argumentation Frameworks
We propose and investigate a simple ranking-measure-based extension semantics for abstract argumentation frameworks based on their generic instantiation by default knowledge bases and the ranking construction semantics for default reasoning. In this context, we consider the path from structured to logical to shallow semantic instantiations. The resulting well-justified JZ-extension semantics diverges from more traditional approaches.
Default Reasoning and the Transferable Belief Model
Smets, Philippe, Hsia, Yen-Teh
Inappropriate use of Dempster's rule of combination has led some authors to reject the Dempster-Shafer model, arguing that it leads to supposedly unacceptable conclusions when defaults are involved. A most classic example is about the penguin Tweety. This paper will successively present: the origin of the miss-management of the Tweety example; two types of default; the correct solution for both types based on the transferable belief model (our interpretation of the Dempster-Shafer model (Shafer 1976, Smets 1988)); Except when explicitly stated, all belief functions used in this paper are simple support functions, i.e. belief functions for which only one proposition (the focus) of the frame of discernment receives a positive basic belief mass with the remaining mass being given to the tautology. Each belief function will be described by its focus and the weight of the focus (e.g. m(A)=.9). Computation of the basic belief masses are always performed by vacuously extending each belief function to the product space built from all variables involved, combining them on that space by Dempster's rule of combination, and projecting the result to the space corresponding to each individual variable.
Towards Solving the Multiple Extension Problem: Combining Defaults and Probabilities
The multiple extension problem arises frequently in diagnostic and default inference. That is, we can often use any of a number of sets of defaults or possible hypotheses to explain observations or make Predictions. In default inference, some extensions seem to be simply wrong and we use qualitative techniques to weed out the unwanted ones. In the area of diagnosis, however, the multiple explanations may all seem reasonable, however improbable. Choosing among them is a matter of quantitative preference. Quantitative preference works well in diagnosis when knowledge is modelled causally. Here we suggest a framework that combines probabilities and defaults in a single unified framework that retains the semantics of diagnosis as construction of explanations from a fixed set of possible hypotheses. We can then compute probabilities incrementally as we construct explanations. Here we describe a branch and bound algorithm that maintains a set of all partial explanations while exploring a most promising one first. A most probable explanation is found first if explanations are partially ordered.
Some Properties of Plausible Reasoning
This paper presents a plausible reasoning system to illustrate some broad issues in knowledge representation: dualities between different reasoning forms, the difficulty of unifying complementary reasoning styles, and the approximate nature of plausible reasoning. These issues have a common underlying theme: there should be an underlying belief calculus of which the many different reasoning forms are special cases, sometimes approximate. The system presented allows reasoning about defaults, likelihood, necessity and possibility in a manner similar to the earlier work of Adams. The system is based on the belief calculus of subjective Bayesian probability which itself is based on a few simple assumptions about how belief should be manipulated. Approximations, semantics, consistency and consequence results are presented for the system. While this puts these often discussed plausible reasoning forms on a probabilistic footing, useful application to practical problems remains an issue.